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Abstract 


We propose Neural Responding Machine (NRM), a neural network-based response generator for 
Short-Text Conversation. NRM takes the general encoder-decoder framework: it formalizes the 
generation of response as a decoding process based on the latent representation of the input text, 
while both encoding and decoding are realized with recurrent neural networks (RNN). The NRM 
is trained with a large amount of one-round conversation data collected from a microblogging 
service. Empirical study shows that NRM can generate grammatically correct and content-wise 
appropriate responses to over 75% of the input text, outperforming state-of-the-arts in the same 
setting, including retrieval-based and SMT-based models. 


1 Introduction 


Natural language conversation is one of the most challenging artificial intelligence problems, 
which involves language understanding, reasoning, and the utilization of common sense knowl¬ 
edge. Previous works in this direction mainly focus on either rule-based or learning-based meth¬ 


ods (Williams and Young, 2007 ; |Sch atzmann et al., 2006 [ [Misu et al., 2012[|Litman et al., 2000| ). These 
types of methods often rely on manual effort in designing rules or automatic training of model with a par¬ 
ticular learning algorithm and a small amount of data, which makes it difficult to develop an extensible 
open domain conversation system. 

Recently due to the explosive growth of microblogging services such as TwitteiQ and WeibcH, the 
amount of conversation data available on the web has tremendously increased. This makes a data-driven 
approach to attack the conversation problem ( jJi et al., 2014||Ritter et al., 2011 [ ) possible. Instead of mul¬ 
tiple rounds of conversation, the task at hand, referred to as Short-Text Conversation (STC), only consid¬ 
ers one round of conversation, in which each round is formed by two short texts, with the former being 
an input (referred to as post) from a user and the latter a response given by the computer. The research 
on STC may shed light on understanding the complicated mechanism of natural language conversation. 

Previous methods for STC fall into two categories, 1) the retrieval-based method ( Ji et al., 2014| , 
and 2) the statistical machine translation (SMT) based method ( jRitter et al., 201 1[ ). The basic idea of 
retrieval-based method is to pick a suitable response by ranking the candidate responses with a linear or 
non-linear combination of various matching features (e.g. number of shared words). The main drawbacks 
of the retrieval-based method are the following 

• the responses are pre-existed and hard to be customized for the particular text or requirement from 
the task, e.g., style or attitude. 


• the use of matching features alone is usually not sufficient for distinguishing positive responses from 
negative ones, even after time consuming feature engineering, (e.g., a penalty due to mismatched 
named entities is difficult to be incorporated into the model) 

The SMT-based method, on the other hand, is generative. Basically it treats the response generation as 
a translation problem, in which the model is trained on a parallel corpus of post-response pairs. Despite 


1 https ://twitter.com/. 
2 http://www. weibo.com/. 
















its generative nature, the method is intrinsically unsuitable for response generation, because the responses 
are not semantically equivalent to the posts as in translation. Actually one post can receive responses 
with completely different content, as manifested through the example in the following figure: 


Post 

Having my fish sandwich right now 

User A 

UserB 

UserC 

For god’s sake, it is 11 in the morning 
Enhhhh... sounds yummy 
which restaurant exactly? 


1.1 Overview 

In this paper, we take a probabilistic model to address the response generation problem, and propose 
employing a neural encoder-decoder for this task, named Neural Responding Machine (NRM). The 
neural encoder-decoder model, as illustrated in Figure |T] first summarizes the post as a vector repre¬ 
sentation, then feeds this representation to decoder to generate responses. We further generalize this 
scheme to allow the post representation dynamically change during the generation process, following the 
idea in ( jBahdanau et ah, 20141 ) originally proposed for neural-network-based machine translation with 
automatic alignment. 


For god's sake, it is 11 in the morning 

If , 


Enhhhh... sounds yummy ^ Tt j-, which restaurant exactly? 



Having my fish sandwich right now 


Figure 1: The diagram of encoder-decoder framework for automatic response generation. 


NRM essentially estimates the likelihood of a response given a post. Clearly the estimated probability 
should be complex enough to represent all the suitable responses. Similar framework has been used 
for machine translation with a remarkable success (Kalchbrenner and Blunsom, 2013j [Auli et ah, 2013 


Sutskever et al., 2014 Bahdanau et al., 2014). Note that in machine translation, the task is to estimate 


the probability of a target language sentence conditioned on the source language sentence with the same 
meaning, which is much easier than the task of STC which we are considering here. In this paper, we 
demonstrate that NRM, when equipped with a reasonable amount of data, can yield a satisfying estimator 
of responses (hence response generator) for STC, despite the difficulty of the task. 

Our main contributions are two-folds: 1) we propose to use an encoder-decoder-based neural net¬ 
work to generate a response in STC; 2) we have empirically verified that the proposed method, when 
trained with a reasonable amount of data, can yield performance better than traditional retrieval-based 
and translation-based methods. 


1.2 RoadMap 

In the remainder of this paper, we start with introducing the dataset for STC in Section [2] Then we 
elaborate on the model of NRM in Section [3] followed by the details on training in Section |4] After that, 
we report the experimental results in Section[5] In Section |6]we conclude the paper. 

2 The Dataset for STC 

Our models are trained on a corpus of roughly 4.4 million pairs of conversations from Weibod 


3 The dataset and its English translation (by machine translation system) will be released soon. 























2.1 Conversations on Sina Weibo 


Weibo is a popular Twitter-like microblogging service in China, on which a user can post short messages 
(referred to as post in the reminder of this paper) visible to the public or a group of users following 
her/him. Other users make comment on a published post, which will be referred to as response. Just like 
Twitter, Weibo also has the length limit of 140 Chinese characters on both posts and responses, making 
the post-response pair an ideal surrogate for short-text conversation. 


Training 

#posts 

219,905 

#responses 

4,308,211 

#pairs 

4,435,959 

Test Data 

#test posts 

110 

Labeled Dataset 

(retrieval-based) 

#posts 

225 

#responses 

6,017 

#labeled pairs 

6,017 

Fine Tuning 

(SMT-based) 

#posts 

2,925 

#responses 

3,000 

#pairs 

3,000 


Table 1: Some statistics of the dataset. Labeled Dataset and Fine Tuning are used by retrieval-based 
method for learning to rank and SMT-based method for fine tuning, respectively. 


2.2 Dataset Description 

To construct this million scale dataset, we first crawl hundreds of millions of post-response pairs, and 


then clean the raw data in a similar way as suggested in (Wang et al., 2013), including 1) removing trivial 
responses like “wow”, 2) filtering out potential advertisements, and 3) removing the responses after first 
30 ones for topic consistency. Table Q] shows some statistics of the dataset used in this work. It can 
be seen that each post have 20 different responses on average. In addition to the semantic gap between 
post and its responses, this is another key difference to a general parallel data set used for traditional 
translation. 

3 Neural Responding Machines for STC 

The basic idea of NRM is to build a hidden representation of a post, and then generate the response 
based on it, as shown in Figure [2] In the particular illustration, the encoder converts the input sequence 
x = (xi, • • • , xt) into a set of high-dimensional hidden representations h = (h\, • • • , hj), which, along 
with the attention signal at time t (denoted as at), are fed to the context-generator to build the context 
input to decoder at time t (denoted as c/ ). Then cy is linearly transformed by a matrix L (as part of the 
decoder) into a stimulus of generating RNN to produce the f-th word of response (denoted as yt). 

In neural translation system, L converts the representation in source language to that of target lan¬ 
guage. In NRM, L plays a more difficult role: it needs to transform the representation of post (or some 
part of it) to the rich representation of many plausible responses. It is a bit surprising that this can be 
achieved to a reasonable level with a linear transformation in the “space of representation”, as validated 
in Section 1531 where we show that one post can actually invoke many different responses from NRM. 

The role of attention signal is to determine which part of the hidden representation h should be em¬ 
phasized during the generation process. It should be noted that at could be fixed over time or changes 
dynamically during the generation of response sequence y. In the dynamic settings, at can be function 
of historically generated subsequence (yi, ■ ■ ■ ,y t - 1 ), input sequence x or their latent representations, 
more details will be shown later in Section [3721 

We use Recurrent Neural Network (RNN) for both encoder and decoder, for its natural ability to 
summarize and generate word sequence of arbitrary lengths (Mikolo v et al., 20T0l|Sutskever et al., 201 4[ 


Cho et al., 2014 1 . 
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Figure 2: The general framework and dataflow of the encoder-decoder-based NRM. 



Figure 3: The graphical model of RNN decoder. The dashed lines denote the variables related to the 
function g(-), and the solid lines denote the variables related to the function /(•). 


3.1 The Computation in Decoder 

Figure [3] gives the graphical model of the decoder, which is essentially a standard RNN language model 
except conditioned on the context input c. The generation probability of the t-th word is calculated by 

p(yt\yt- 1 ,-" > 2 / 1 , x) = g(y t -i,s t ,ct), (1) 

where y t is a one-hot word representation, g(-) is a softmax activation function, and st is the hidden state 
of decoder at time t calculated by 


■St — 


( 2 ) 


and /(•) is a non-linear activation function and the transformation L is often assigned as pa¬ 
rameters of /(•). Flere /(•) can be a logistic function, the sophisticated long short-term mem¬ 
ory (LSTM) unit dHochreiter and Schmidhuber, 1997[ ), or the recently proposed gated recurrent 
unit (GRU) (Chung et al., 2014 ; [Cho et al., 2014| ). Compared to “ungated” logistic function, LSTM and 
GRU are specially designed for its long term memory: it can store information over extended time steps 
without too much decay. We use GRU in this work, since it performs comparably to LSTM on squence 
modeling ([Chung et al., 2014]), but has less parameters and easier to train. 


3.2 The Computation in Encoder 

We consider three types of encoding schemes, namely 1) the global scheme, 2) the local scheme, and the 
hybrid scheme which combines 1) and 2). 

Global Scheme: Figure |4|shows the graphical model of the RNN-encoder and related context generator 
for a global encoding scheme. The hidden state at time t is calculated by h t = f(xt, ht~ i) (i.e. still 





































GRU unit), and with a trivial context generation operation, we essentially use the final hidden state hr 
as the global representation of the sentence. The same strategy has been taken in ( jCho et al., 20141 ) and 
( jSutskever et ah, 2014) for building the intermediate representation for machine translation. This scheme 
however has its drawbacks: a vectorial summarization of the entire post is often hard to obtain and may 
lose important details for response generation, especially when the dimension of the hidden state is not 
big cnougfQ. In the reminder of this paper, a NRM with this global encoding scheme is referred to as 
NRM-glo. 



Figure 4: The graphical model of the encoder in NRM-glo, where the last hidden state is used as the 
context vector q = hr- 


Local Scheme: Recently, Bahda nau et al. (2014 ) and |Graves (2013 ) introduced an attention mecha¬ 
nism that allows the decoder to dynamically select and linearly combine different parts of the input 
sequence q = Yjj=i a tjhj, where weighting factors atj determine which part should be selected to 
generate the new word y t , which in turn is a function of hidden states atj = q(hj , St- 1 ), as pictorially 
shown in Figure [5] Basically, the attention mechanism atj models the alignment between the inputs 
around position j and the output at position t, so it can be viewed as a local matching model. This local 
scheme is devised in dBahdanau et al., 20141 ) for automatic alignment between the source sentence and 
the partial target sentence in machine translation. This scheme enjoys the advantage of adaptively focus¬ 
ing on some important words of the input text according to the generated words of response. A NRM 
with this local encoding scheme is referred to as NRM-loc. 



Figure 5: The graphical model of the encoder in NRM-loc, where the weighted sum of hidden sates is 
used as the context vector q = J2j=i a tjhj- 


jSutskever et al. (2014) has to use 4,000 dimension for satisfying performance on machine translation, while 
fCho et al., 20141) with a smaller dimension perform poorly on translating an entire sentence. 





















































3.3 Extensions: Local and Global Model 

In the task of STC, NRM-glo has the summarization of the entire post, while NRM-loc can adaptively 
select the important words in post for various suitable responses. Since post-response pairs in STC are 
not strictly parallel and a word in different context can have different meanings, we conjecture that the 
global representation in NRM-glo may provide useful context for extracting the local context, therefore 
complementary to the scheme in NRM-loc. It is therefore a natural extension to combine the two models 
by concatenating their encoded hidden states to form an extended hidden representation for each time 
stamp, as illustrated in Figure [6] We can see the summarization hj, is incorporated into ct and aitj to 
provide a global context for local matching. With this hybrid method, we hope both the local and global 
information can be introduced into the generation of response. The model with this context generation 
mechanism is denoted as NRM-hyb. 

It should be noticed that the context generator in NRM-hyb will evoke different encoding mechanisms 
in the global encoder and the local encoder, although they will be combined later in forming a unified 
representation. More specifically, the last hidden state of NRM-glo plays a role different from that of 
the last state of NRM-loc, since it has the responsibility to encode the entire input sentence. This role 
of NRM-glo, however, tends to be not adequately emphasized in training the hybrid encoder when the 
parameters of the two encoding RNNs are learned jointly from scratch. For this we use the following 
trick: we first initialize NRM-hyb with the parameters of NRM-loc and NRM-glo trained separately, 
then fine tune the parameters in encoder along with training the parameters of decoder. 



local encoder 


Figure 6: The graphical model for the encoder in NRM-hyb, while context generator function is ct = 
Ylj=i a tj[h l j', hj], here [h ^; h 9 T \ denotes the concatenation of vectors h!- and h? T 

To learn the parameters of the model, we maximize the likelihood of observing the original response 
conditioned on the post in the training set. For a new post, NRMs generate their responses by using a 
left-to-right beam search with beam size =10. 

4 Experiments 

We evaluate three different settings of NRM described in Section [3) namely NRM-glo, NRM-loc, and 
NRM-hyb, and compare them to retrieval-based and SMT-based methods. 

4.1 Implementation Details 

We use Stanford Chinese word segmenter 0 to split the posts and responses into sequences of words. 
Although both posts and responses are written in the same language, the distributions on words for the 
two are different: the number of unique words in post text is 125,237, and that of response text is 679,958. 
We therefore construct two separate vocabularies for posts and responses by using 40,000 most frequent 
words on each side, covering 97.8% usage of words for post and 96.2% for response respectively. All the 


s http ://nlp. Stanford, edu/software/segmenter. shtml 































remaining words are replaced by a special token “UNK”. The dimensions of the hidden states of encoder 
and decoder are both 1,000, and the dimensions of the word-embedding for post and response are both 
620. Model parameters are initialized by randomly sampling from a uniform distribution between -0.1 
and 0.1. All our models were trained on a NVIDIA Tesla K40 GPU using stochastic gradient descent 
algorithm with mini-batch. The training stage of each model took about two weeks. 


4.2 Competitor Models 

Retrieval-based: with retrieval-based models, for any given post p*, the response r* is retrieved from 
a big post-response pairs (p, r ) repository. Such models rely on three key components: a big repository, 
sets of feature functions 4>j(p*, (p, r)), and a machine learning model to combine these features. In this 
work, the whole 4.4 million Weibo pairs are used as the repository, 14 features, ranging from simple 
cosine similarity to some deep matching models ( jJi et al., 2014| ) are used to determine the suitability of 
a post to a given post p* through the following linear model 


score(p*,(p,r)) = ( p,r )). 


(3) 


Following the ranking strategy in (Ji et al., 2014), we pick 225 posts and about 30 retrieved responses for 
each of them given by a baseline retrieve^ from the 4.4M repository, and manually label them to obtain 
labeled 6,017 post-response pairs. We use ranking SVM model ( (Joachims, 2006| ) for the parameters u! r 
based on the labeled dataset. In comparison to NRM, only the top one response is considered in the 
evaluation process. 

SMT-based: In SMT-based models, the post-response pairs are directly used as parallel data for train¬ 
ing a translation model. We use the most widely used open-source phrase-based translation model- 


Moses (Koehn et aL, 2007). Another parallel data consisting of 3000 post-response pairs is used to tune 


the system. In {Ritter et al., 2011), the authors used a modified SMT model to obtain the “Response” 
of Twitter “Stimulus”. The main modification is in replacing the standard GIZA++ word alignment 


model (Och and Ney, 2003) with a new phrase-pair selection method, in which all the possible phrase- 
pairs in the training data are considered and their associated probabilities are estimated by the Fisher’s 
Exact Test, which yields performance slightly better than default setting. Compared to retrieval-based 
methods, the generated responses by SMT-based methods often have fluency or even grammatical prob¬ 
lems. In this work, we choose the Moses with default settings as our SMT model. 


5 Results and Analysis 

Automatic evaluation of response generation is still an open problem. The widely accepted evaluation 
methods in translation (e.g. BLUE score ( Papineni et al., 2002] )) do not apply, since the range of the 
suitable responses is so large that it is practically impossible to give reference with adequate coverage. It 
is also not reasonable to evaluate with Peiplexity, a generally used measurement in statistical language 
modeling, because the naturalness of response and the relatedness to post can not be well evaluated. 
We therefore resort to human judgement, similar to that taken in ( |Ritter et al., 2011[ ) but with important 
difference. 


5.1 Evaluation Methods 

We adopt human annotation to compare the performance of different models. Five labelers with at least 
three-year experience of Sina Weibo are invited to do human evaluation. Responses obtained from the 
five evaluated models are pooled and randomly permuted for each labeler. The labelers are instructed 
to imagine that they were the authors of the original posts and judge whether a response (generated or 
retrieved) is appropriate and natural to a input post. Three levels are assigned to a response with scores 
from 0 to 2: 

6 we use the default similarity function of LuceneQ 

8 Reported results showed that the new model outperformed the baseline SMT model 57.7% of the time. 















Post 

There are always 8 players at the Italian restricted area. Unbelievable! 

Related Criterion 

Labels 

Responsel 

I am a big fan of the Italy team, waiting for the football game to start 

Logic Consistency 

Unsuitable 

Response2 

Italian food is absolutely delicious. 

Semantic Relevance 

Unsuitable 

Response3 

AA^T’A! 

Unbelievable! 

Generality 

Neutral 

Response4 

Ha!Ha!Ha! it is still 0:0, no goal so far. 

Scenario Dependence 

Neutral 

Responses 

Af'J 

This is exactly the Italian defending style football game. 


Suitable 


Figure 7: An example post and its five candidate responses with human annotation. The content of the 
post implies that the football match is already started, while the author of Response 1 is still waiting for 
the match to start. Response2 talks about the food of Italy. Response3 is a widely used response, but it 
is suitable to this post. Response4 states that the current score is still 0:0, it is a suitable response only in 
this specific scenario. 


Models 

Mean Score 

Suitable (+2) 

Neutral (+1) 

Unsuitable (0) 

Agreement 

NRM-glo 

0.969 

34.0% 

28.9% 

37.1% 

0.397 

NRM-loc 

1.065 

36.0% 

34.5% 

29.5% 

0.247 

NRM-hyb 

1.156 

39.3% 

37.1% 

23.6% 

0.309 

Rtr.-based 

0.984 

29.8% 

38.7% 

31.5% 

0.346 

SMT-based 

0.313 

5.6% 

20.0% 

74.4% 

0.448 


Table 2: The results of evaluated methods. Mean score is the average value of annotated scores over all 
annotations. (Rtr.-based means the retrieval-based method) 


ESI 

Post 

NRM-glo 

NRM-loc 

NRM-hyb 

Rtr.-based 

■ 

A A A 3- ftp A A , $i o 

##-4, IfS&&A! 


/£- 1 

I* 'Rfc ... 


High fever attacks me 
every New Year’s Day 

Rest more to boost 
your immune system. 

Go to see a doctor 
every New Year’s Day 

ar-. 

Get well soon and stay 
healthy! 

High fever... 

2 

AA T— $ , iSifAfn 
fe, iA 

I gain one more year. 
Grateful to my group, so 
happy 

A El 'ft ,f,! fcAA# 

if* ! 

Happy birthday! Will 
stand by you forever! 

:£> A: | 

/Ay 9C 1 

Happy birthday! Wish 
you always stay young 
and pretty! 

AT AT, f H 

Alf! 

Getting old now. Time 
has no mercy. 

ARAAT-^ 

I just gain only one 
more year. 

3 

First day of being a 
vegetarian. Hold on, 

CuiDuoLa 

A A#! 410;*! 

I come to offer my 
support. Keep it up! 

!£4H±, !£r#T-£-„ 

Hold on, keep it up. 

I have been being a 
vegetarian everyday 

!Sr#f±! 

Hold on! 


i£ f t A A A & it, 4j? A A 
4% tb A A A A vCi 

We should go out with 
some cute guys to enjoy 
a great outing in such a 
nice weather. 

ilAAIUMf? A k\ 

Where is it? It is so 
beautiful! 

Pa JtmM, -tf'liNf#. 

Such a nice sunny day! 

I am in a great mood. 

ilf+AAA-TAJ-itto 

It is indeed a very nice 
weather. 

Adj'lf tbA^fA-f- 
#UIA 'A'A 

WenShan, let’s go out 
to get some 
inspiration. Ha! Ha! 


Figure 8: Some responses generated by different models (originally in Chinese with their literal English 
translation), where the words in boldfaces arc entity names. 





































































• Suitable (+2): the response is evidently an appropriate and natural response to the post; 

• Neutral (+1): the response can be a suitable response in a specific scenario; 

• Unsuitable (0): it is hard or impossible to find a scenario where response is suitable. 

To make the annotation task operable, the suitability of generated responses is judged from the follow¬ 
ing five criteria: 

(a) Grammar and Fluency: Responses should be natural language and free of any fluency or grammat¬ 
ical errors; 

(b) Logic Consistency: Responses should be logically consistent with the test post; 

(c) Semantic Relevance: Responses should be semantically relevant to the test post; 

(d) Scenario Dependence: Responses can depend on a specific scenario but should not contradict the 
first three criteria; 

(e) Generality: Responses can be general but should not contradict the first three criteria; 

If any of the first three criteria (a), (b), and (c) is contradicted, the generated response should be labeled 
as “Unsuitable”. The responses that are general or suitable to post in a specific scenario should be 
labeled as “Neutral”. Figure |7] shows an example of the labeling results of a post and its responses. The 
first two responses are labeled as “Unsuitable” because of the logic consistency and semantic relevance 
errors. Response4 depends on the scenario (i.e., the current score is 0:0), and is therefore annotated as 
“Neutral”. 


Model A 

Model B 

Average 

rankings 

p value 

NRM-loc 

NRM-glo 

(1.463, 1.537) 

2.01% 

NRM-hyb 

NRM-glo 

(1.434, 1.566) 

0.01% 

NRM-hyb 

NRM-loc 

(1.465, 1.535) 

3.09% 

Rtr.-based 

NRM-glo 

(1.512, 1.488) 

48.1% 

Rtr.-based 

NRM-loc 

(1.533, 1.467) 

6.20% 

Rtr.-based 

NRM-hyb 

(1.552, 1.448) 

0.32% 

SMT 

NRM-hyb 

(1.785, 1.215) 

0.00 % 

SMT 

Rtr.-based 

(1.738, 1.262) 

0.00 % 


Table 3: p -values and average rankings of Friedman test for pairwise model comparison. (Rtr.-based 
means the retrieval-based method) 


5.2 Results 


Our test set consists of 110 posts that do not appear in the training set, with length between 6 to 22 
Chinese words and 12.5 words on average. The experimental results based on human annotation are 
summarized in Table [2j consisting of the ratio of three categories and the agreement among the five 
labelers for each model. The agreement is evaluated by Fleiss’ kappa flFleiss, 1971| >, as a statistical 
measure of inter-rater consistency. Except the SMT-based model, the value of agreement is in a range 
from 0.2 to 0.4 for all the other models, which should be interpreted as “Fair agreement”. The SMT-based 
model has a relatively higher kappa value 0.448, which is larger than 0.4 and considered as “Moderate 
agreement”, since the responses generated by the SMT often have the fluency and grammatical errors, 
making it easy to reach an agreement on such unsuitable cases. 

From Table [2j we can see the SMT method performs significantly worse than the retrieval-based and 
NRM models and 74.4% of the generated responses were labeled as unsuitable mainly due to fluency 
and relevance errors. This observation confirms with our intuition that the STC dataset, with one post 
potentially corresponding to many responses, can not be simply taken as parallel corpus in a SMT model. 
Surprisingly, more than 60% of responses generated by all the three NRM are labeled as “Suitable” 
or “Neutral”, which means that most generated responses are fluent and semantically relevant to post. 
Among all the NRM variants 



















• NRM-loc outperforms NRM-glo, suggesting that a dynamically generated context might be more ef¬ 
fective than a “static” fixed-length vector for the entire post, which is consistent with the observation 
made in ( [Bahdanau et ah, 2014] ) for machine translation; 

• NRM-hyp outperforms NRM-loc and NRM-glo, suggesting that a global representation of post is 
complementary to dynamically generated local context. 

The retrieval-based model has the similar mean score as NRM-glo, and its ratio on neutral cases 
outperforms all the other methods. This is because 1) the responses retrieved by retrieval-based method 
are actually wrote by human, so they do not suffer from grammatical and fluency problems, and 2) 
the combination of various feature functions potentially makes sure the picked responses semantically 
relevant to test posts. However the picked responses are not customized for new test posts, so the ratio 
of suitable cases is lower than the three neural generation models. 

To test statistical significance, we use the Friedman test (Howell, 2010j ), which is a non-parametric 
test on the differences of several related samples, based on ranking. Table [3] shows the average rankings 
over all annotations and the corresponding p -values for comparisons between different pairs of methods. 
The comparison between retrieval-based and NRM-glo is not significant and their difference in ranking 
is tiny. This indicates that the retrieval-based method is comparable to the NRM-glo method. The NRM- 
hyb outperforms all the other methods, and the difference is statistically significant (p < 0.05). The 
difference between NRM-loc and retrieval-based method is marginal (p = 0.062). SMT is significantly 
worse than retrieval-based and NRM-hyb methods. 

5.3 Case Study 

Figure [8] shows some example responses generated by our NRMs (only the one with biggest likelihood 
is given) and the comparable retrieval-based model. It is intriguing to notice that three NRM variants 
give suitable but quite distinct responses, with different perspectives and choices of words. This, as we 
conjecture, is caused by both the architecture variations among models as well as the variations from 
random effects like the initialization of parameters. Another interesting observation is on the forth exam¬ 
ple, where the retrieval-based method returns a response with the a mismatched entity name “WenShan”, 
which is actually a quite common problem for retrieval-based model, where the inconsistency details 
(e.g., dates, named entities), which often render the response unsuitable, cannot be adequately consid¬ 
ered in the matching function employed in retrieving the responses. In contrast, we observe that NRMs 
tend to make general response and barely generate those details. 


Post 

!£#f±. 

First day of being a vegetarian. Hold on, CuiDuoLa 

R1 

lir#t£, $■#**! 

Hold on, Keep up your vegetarian diet. 

R2 

Would you like to be healthy? Would you like to live 
long? If so, follow me on Weibo. 

R3 

This is the so-called vegetarian diet ? 

R4 

Eat up so that you will have enough energy to go for 
a diet! 

R5 

Health is the essence for winning your future battles. 
No rush! 


Figure 9: Multiple responses generated by the NRM-hyb. 

We also use the NRM-hyb as an example to investigate the ability of NRM to generate multiple 
responses. Figure |9]lists 5 responses to the same post, which are gotten with beam search with beam size 













= 500, among which we keep only the best one (biggest likelihood) for each first word. It can be seen 
that the responses arc fluent, relevant to the post, and still vastly different from each other, validating our 
initial conjecture that NRM, when fueled with large and rich training corpus, could work as a generator 
that can cover a lot of modes in its density estimation. 

6 Conclusions and Future Work 

In this paper, we explored using encoder-decoder-based neural network system, with coined name Neural 
Responding Machine, to generate responses to a post. Empirical studies confirm that the newly proposed 
NRMs, especially the hybrid encoding scheme, can outperform state-of-the-art retrieval-based and SMT- 
based methods. Our preliminary study also shows that NRM can generate multiple responses with great 
variety to a given post. In future work, we would consider adding the intention (or sentiment) of users as 
an external signal of decoder to generate responses with specific goals. 
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